{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.决策树\n",
    "<P>决策树的id3算法划分依据为信息增益,信息增益=初始信息熵-在非分类属性下的信息熵</P>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集精度:1.0\n",
      "测试集精度:0.9370629370629371\n"
     ]
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.datasets import load_breast_cancer\n",
    "from sklearn.model_selection import train_test_split\n",
    "cancer = load_breast_cancer()\n",
    "X_train,X_test,y_train,y_test = train_test_split(cancer.data,cancer.target,random_state=42,stratify=cancer.target) \n",
    "tree = DecisionTreeClassifier(random_state=0) # 固定树的random_state，用于在内部解决平局问题\n",
    "tree.fit(X_train,y_train)\n",
    "print(f\"训练集精度:{tree.score(X_train,y_train)}\")\n",
    "print(f\"测试集精度:{tree.score(X_test,y_test)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<P>有结果可知,数据拟合度较高。这是因为叶子节点都是纯的，导致数据过拟合。</P>\n",
    "<P>解决方法为:使用预剪枝(主动限制树的最大深度,主动限制叶子节点的个数</P>\n",
    "<P>下面我们将限制树的最大深度(设置max_depth=4),这样只能问四个问题,能够减小过拟合的问题</P>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集精度:0.9882629107981221\n",
      "测试集精度:0.951048951048951\n"
     ]
    }
   ],
   "source": [
    "tree = DecisionTreeClassifier(random_state=0,max_depth=4) # 固定树的random_state，用于在内部解决平局问题\n",
    "tree.fit(X_train,y_train)\n",
    "print(f\"训练集精度:{tree.score(X_train,y_train)}\")\n",
    "print(f\"测试集精度:{tree.score(X_test,y_test)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 分析决策树"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import export_graphviz # 可视化决策树的方法\n",
    "export_graphviz(tree,out_file='tree.dot',class_names=['malignant','benign'],feature_names=cancer.feature_names,impurity=False,filled=True)\n",
    "# 之后在命令行运行dot -Tpdf tree.dot -o tree.pdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Feature importances:[0.         0.         0.         0.         0.         0.\n",
      " 0.         0.         0.         0.         0.01019737 0.04839825\n",
      " 0.         0.         0.0024156  0.         0.         0.\n",
      " 0.         0.         0.72682851 0.0458159  0.         0.\n",
      " 0.0141577  0.         0.018188   0.1221132  0.01188548 0.        ]\n"
     ]
    }
   ],
   "source": [
    "print(f\"Feature importances:{tree.feature_importances_}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAegAAAEGCAYAAACuHgb+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydebxWVdXHvz/QUEAxldfXTCLnCUHBqVRwyDI1Z9EwRStTU7JSs8lMszQ1pxxSUxwoDaeccohAcAaUUU1TKVNfwwlFJRHW+8daD/fc5z7Pc58L93IvsL6fz/O55+yzz977HPy4zt57/daSmZEkSZIkSceiU3sPIEmSJEmSpqSBTpIkSZIOSBroJEmSJOmApIFOkiRJkg5IGugkSZIk6YAs194DSJYeVl99devdu3d7DyNJkmSJYuLEiW+YWc/y8jTQSavRu3dvJkyY0N7DSJIkWaKQ9M9K5bnEnSRJkiQdkDTQSZIkSdIBSQOdJEmSJB2QNNBJkiRJ0gFJA50kSZIkHZA00EmSJEnSAUkDnSRJkiQdkDTQSZIkSdIBWWINtKR9JG1S5VpPSY9LekrSDovYT29JX62z3rQ66g2XdEAcX1XtGdoSSUdLOmxx95skSZLUT4c30JI6V7m0D1DNuO0CPGtmW5jZuDrbq0ZvoFkDvTCY2TfM7Om2aLuZfi83s+sWd79JkiRJ/bSZgZZ0sqRhcXy+pL/F8S6SbojjQyRNlTRN0tmFe2dLOl3S48B2ks6S9LSkKZLOlfQ54CvAOZImSVq3cG8/4NfAl+PaihXaO1XS+Oj3CkmKe9eT9FdJkyU9Ge2eBewQbX03Zsrj4vqTMZZa70GSfhvjvxv4n8K1MZIGFJ75bEkTYwxbx/UXJX0l6nSWdE6MfYqkb0X5oKh7s6RnJY0oPFOjdxdlp0k6sfS+JD0W12+T9MnC2M6W9ISk5xZ1JSJJkiRpIWbWJj9gW2BkHI8DngCWB34GfAv4FPAvoCceE/xvwD5R34CD4nhV4O+A4nyV+DscOKBK30OB3xbOF7RXarNwfD2wVxw/DuwbxysAXYFBwF2F+l2BFeJ4fWBCHPcGplUYy37AA0DneOZ3SuMGxgADCmPcPY5vA+6P99UXmBTlRwE/ieMuwATgszHGWcCn8Y+uR4Hta7y704AT43gKMDCOTwcuKIztvDj+MvDXKu/6qBjHhF69elmSJEnSMkp2pPzXlkvcE4H+klYC/htGYwCwA26wtwLGmNlMM/sYGAHsGPfOA26J43eBOcBVkvYDPliIsRTbA9gp9qinAjsDm8Y41zKz2wDMbI6ZVepreeDKuHck1ZfZS+wI/NHM5pnZq/iHSCU+Au6N46nAg2Y2N457R/luwGGSJuEfE6vhHwkAT5jZv81sPjAp7qn57iT1wI32g1F0LQ3/BgC3xt+JhTE0wsyuMLMBZjagZ88myViSJEmShaTNDHQYlxnAEcAjuFHeCVgXeAZQjdvnmNm8aOdjYGvcwO5DgxFrCQvak7QCcCk+i+0DXInPlmuNp8h3gdfxme0A4BN13GN11JkbX1IA8/GPGsLglrKOCTjezPrF77Nmdn9c+2+hrXnAcq3w7kptziMznyVJkixW2tpJbCxwYvwdBxyNL9caPgMcKGn1cNw6BHiwvAFJ3YEeZnYPcALQLy69B6y0EGNaIf6+EW0fAGBm7wL/lrRP9NtFUtcK/fQAXgvD+TV86brEShW8sscCB8f+8Zr4R8rCch9wjKTlY4wbSOpWrXKNdwf+HoYAbxf2l79GhX+DJEmSZPHT1rOiccCPgUfN7H1Jc6IMM3tN0g+B0fjM8B4z+3OFNlYC/hwzX+EzWIAb8aXmYfhs+IV6BmRm70i6El86ngGML1z+GvA7SacDc4ED8T3ajyVNxve9LwVukXRgjP39wv0r40veRc/s2/Bl9KnAcyyaAbwKX2p+MpzAZuIz42pUe3cAKwLHAocCl8fHyIvAEWq5pzsAU1+ZRe9T7m5UNuOsPRamqSRJkmUeNayqLv1IOhlf7r5I0vlAXzPbWdIuwBFmdqikQ4Af4QbtbjP7Qdw7G/gN8EXg+8CeuCf5x7hD163AXbiz1ixg/+JHQxj0n+HLxbPMbEdJ4/Al60lR52HgGNyx7LPAmsAGwPdwp7vdgVdwp7a5kmYAf8Bn5cvjDlu/AtYDzjGzy6Pdk4CDcMey28zsZ5JuBPbGncgeAO6O8b2Gz7RvAd4wswujjTOB183somrvt8ua69uah1/QqCwNdJIkSW0kTTSzAeXlHV4H3cqMxZ3UwPePu8dy8fbAOEmfAs7GZ7z9gK1KS95AN9xLext8hrwvsKmZbQ78wsweAe4ATor94fIZ/anAF82sL27YwWfEQ8GXq4EuZjYlrq0L7IEb0RuA0bFn/mGUl3jZzLbDVyaG40v22+Ie2UjaDXck2zqeqb+kHYFTgBdirCdFW1sDPzazTYDfA4dHG52Ag3FHviRJkmQxsKwZ6Pb0LH8YGC7pmzTsW48E9oyPhCNxA1viLwUv7s409vDuXah3R6H8cTN7z8xmAnMkrYJ7fu8GPAU8CWxEg+d3OU+Y2UsAZjYDeFPSFqX7zezN8hskHSVpgqQJ8z6YVcdrSJIkSephmfLMLSwLlzzLp9DYs3yDGrc38iyXtDUesexg4Dh81l2r76MlbYPPfidJ6mdmb0p6AJ8lH4R/LJRY4MUtqdzDe7nyehQ8v8vqCfiVmf2uOB5JvSsM8/2y89IM/3+Bq6s81xXAFeBL3JXqJEmSJC1nmTLQQcmz/Eh81vkbYKKZmTzS2IWSVgfexj3LLy5vILyju5rZPZIeA/4Rl6p6lkta18weBx6XtBewNvAmbgTvBMaZ2Vut+Jwl7gPOkDTCzGZLWgt3gKvHC/42fKl8eeoId9pnrR5MyD3nJEmSVmGZWeJWQ3KNcbjz1aNm9jq+VL3AsxwoeZZPBp6s4Vl+l6QpuFd20bP8JHmSjnXL7jlHEdYU/0iYHH1OxJfMr2m9p20gdNJ/AF6Q9CJwM7BSLFc/LA93ek5UX02F0KVm9hH+Lv5UWj1IkiRJFg9LnRe3pM6VjImk4XjIzpsX/6iqE45pY4CNQlvdnmM5DZhtZqWY3Z3wfesDzez55u6v5MW9pJNe6EmStDUd3otb7ZRcI+5fQ54oYnL8Phfl34u+pkk6Icp6S3pG0pWSpku6X9KKca1Jsg1J3SWNivOpkvaOumdLuhYP2PJj4FRJ349rJ6khIcbPq7yv2ZLOi3ZHSeoZ5dWSXxTTXM6Q9PPCmDaKPemjge/GOzoUl1x9GrhZ0thF+gdOkiRJWkSHMdC0rwTqIjz2dV9gS2C6pP64M9k2uGzpm+HRDO4FfYmZbYonv9g/ykdEeV/gc7iBm4Mn4NgSd0g7T5Lw5fDeZra2mY3EncRG1pBFldMNX4LfEl9m/1mUXwf8IJ59aqG8nDfi3svwxBkzgMuB8+Md3QD8B9eKF6VhjUgv7iRJkrahIxno9pRA7YwbKiKpxSz8w+A2M3vfzGbjgUhKHxAvlYKLxLh7q3qyDQG/jP3qvwJrAWuY2VPA/0j6lKS+wNtm9i/ql0XNB26K4xuA7dV88osizSbCoLI0rBHFZBmdu/ao0kySJEnSUjqMF3d7SqCqUCt5RnliihVr1B+Cp9TsX3jGUjzwm/HAIv+Lz6hL/TaRRdVBS50Jmk2EUU0a1sJ+kiRJkoWgwxjooF0kUMAoPMTmBfI41N1iLMMlnYUbzX3xWN0VMbN3Jf1b0j5mdrukLvisswfwnzDOOwGfKdx2I55Na3VgYJRVlEWZ2X/KuuyEG/cbcQnUQ2Y2S9LbknYws3G0PPnFe3g8caCmNKwiKbNKkiRpPTqagW6v5BrfAa6Q9HV8RnmMmT0ant9PRJ2rzOwpVQ7wUaKUbOMCPOjHXvhS/J2SJuB5mp8tVTaz6bE0/kpIvDCz+yVtDDzqW9XMxhNalBvo9/E81hPx2N+Do/xwypJfVBtsYV8fSf3w7YHDw5HteNxhbH38PY4ipGHVqJQso0h6RCdJktTPUiezWpyoHSVdkmabWfdWbG8oMMDMjlvYNpqTWaWBTpIkaYo6usxqcaKlQNLll1sk6Tq2MIbTJH0/2p8m6RN4xLDBMebBkp5Xg3Srk6R/xPZCkiRJshhYJg00S4ekaxotk3SVlsAhJF2lk4gYdipwU4z5JtwzfEhU2RWYbGZvlL/IlFklSZK0DcuqgU5Jl0u6anE1cFgcH0mVUKQps0qSJGkbOpqT2GIhJV0LJF1VMbOXJb0uaWd8Zj+kuXvSiztJkqT1WFZn0NAg6RqLz5qPBl4HNsbDbw6UtHrIrg6hglwpJF09zOwe4AR8ORzqk3QhqbOklWMM+0jqKqkbvmw+rtrAzexd4N+lZXdJB0jakuYlXQfjRrqS81qlMV+FL3VnsowkSZLFzDJpoMPoVspq1QnYpI2zWn0H2EnSVHzJelMzexIYjku6HickXc08xteAYdH35fje9QhgQEi6hlAm6YrxLpB0lTEa2KTkJBZldwDdaaNMW0mSJEl1ligD3Zre18AXgOdxvfG5wFBgPcL7GnjczPqY2WZmdnLJ+xpP2zhZ0ufC0N2Iv0cBn4zuXomyibgme4SkXSU9DDwEnGlmfYDbgWPjOY4BLjSzzfA97VH4XrSVvLEjy9SLYZRvjn6Ojr5OwvedD8W102/is+L7JZX2s/sBExSJOIAvmtlmktaMsSyPb3u8Gh8xt+LBVkZKKn18JEmSJIuBJW0PeizwfdwTegDQpYr3dX882tj9isheNHhfnyppVeD3eIpHk7SKmb0j6Q6q65dL3tf7hvHqXuZ9LTzi1oPR93rAgcBRwHg82tf2uATrR0DJK3xzfPbbDXhK0t14UJJ9IzrZ6sBjMbZN8EAunzezNyStamZvlY/bHbdZzsy2lvRlPGHGrsDXgVlmtpU80tnDku4H9gPuM7Mz49m6AhfEPV8ws4ckrVLpH0TSUfGM9OrVq5l/viRJkqRelqgZNEue9/VU8xzP04FR5lFhptI4OcWfzezDkDCNxrNYVfTGjjHcXJI7mdlbNcZbKRnGbsBhpRUCYDVcxjUeOEKeD7qPmb2Hy65exbXRX4p31oSiF3fPnj1rDCdJkiRpCUuUgTazucAMGryvx9HY+7qWN3Qj72vcEN6Cz2TvXcgh1et9Pb9wPp/GKxflodyMxt7Y/XDntVLo0npDv1VKhiHg+NA69zOzz5rZ/WY2Fv+QeQW4XtJhZvY20BcYA3wbdxhLkiRJFhNLlIEOKnlfT4rZaYf2vq7C3pJWkLQaMAifzVbzxh4FHBR1iaX65sZd5D7gmNgWQNIGkrpJ+kz0dyW+9L9lLK13MrNbgJ/iQVWSJEmSxcSStgcN7ZBQI+RMlwAnatETapTzBHA30As4w8xelVQtwcZWMY4HJc0D1o2+Go27Rl9X4cvdT8o3qmfiKwiDcK/zubiD2WH4svo1kkofcT9s7kGqJcvIGNxJkiQtJ5NlFNBiTn4Re76zwzu7nvpjgBPNbEKcz8ATXDQJwdkeVEuWkQY6SZKkOlqak2W0pvxKLU9+cWC0OVnS2CgbKul2SXdKeknScfJkGE9JeqywNP2/wPHR122SPhn394t6C8olHYA7xI2IcZSSZhyvhuQYG8X9p0m6WtIYSS+W3k1cO1TSE9HG72K5vrOk4fEcUxWSKknDCu+i2ehjSZIkSeuxVBho2jf5xam4nrgvbshLbIZLq7YGzgQ+MLMtcM/zUozrzwGHRV9TcTkUwHXAD4rlMXufAAyJcXwYdd+I5BiX4XvzJTYCvhj9/0zS8vI804NxmVY/fKl+SLyTtULz3YeGwCSnAFvEOI6u9OKVyTKSJEnahKXFQLen/OphYLikb+JBPUqMNrP3zGwmMAu4M8qn4gkvegCrmFnJie1aYMdq5TX6rySnArjbzP4by9//wWVau+Aa8fEhtdoFWAd4EVhH0sVlkqop+Iz9UODjSp1nsowkSZK2Yakw0O0pvzKzo4GfAGsDk0oe1tQvs1pUKsmpyvsvXRNwbUFmtaGZnVZDUrUH7pTWH5goaUl0KkySJFkiWZr+h1uSXx2Jz1J/A0yMSGGPAxeGdOhtXH51cXkDIb/qamb3SHoM+EdcqipjkrSumT2ORxHbCzfUzWJmsyS9LWkHMxuHx9Z+sFp5c+Ook1G49/r5Zvaf2AtfCXgf+MjMbpH0Ar4i0AlY28xGS3oIX67vjuekrkhms0qSJGk9liYDvdjlV8E5ktaP+qPw5Br9qI/DgcsldcWXmY9opnx4lH+IxxPviqfGrMuL28yelvQTPARqJ2AuPmP+kKaSqs7ADbHkLuB8M6tqnJMkSZLWJWVWSwBtJf+StFws67cKAwYMsAkTJrRWc0mSJMsES7XMqqPSzvKvvSQ9HtKuv0paI8pPk3SFPEnGdZJ6SrpFnuFqvKTPR72tJT0S9z8iacPF8c6SJEkSZ2la4u6ItGf2rYeAbaP+N4CTYyxEf9ub2YeS/oAvXz8kqRceDnRjPHrZjmb2saRdgV8C+5d3osxmlSRJ0iakgW5byuVfT9Ig/xpGQf4FIA/xuSOem7ma/Otu4K46+v40cJM81/MngJcK1+4o6Kh3BTaRFji6rxzj7QFcG/vrhueKboKZXQFcAb7EXce4kiRJkjrIJe42pJ2zb10M/DYCj3wLz4ZV4v3CcSdgu4L0aq1IN3kGruXeDNir7P4kSZKkjUkD3cpI2kfSJoWi1s6+tRH1Zd/qgaePBPcKL29zUOxj3w8cVygvtV28f2itZ06SJElanzTQC0kY1ErsAxQN9DhgTVz+9Tq+VL1A/oVLmkbj8qwna8i/7pI0BViRxvKvk8KRa92ye04DRkoaR2UZ1iA81OgwYEA4nz1NQ0jPXwO/kvQwESEtA5UkSZIsPpY5A724PavNbJSZLR/a7OHA34B9JT0naU8z+wM+I/4LsFO09S0z6x6z3NHAeUD3iIn9WTO7VtIg3HFrGu5Q9k1JQ+SJMKbiDmbrAPvh4Ty7SRoPPADcjBvi7wJ/BX6Lh/18Bt8zH4/ngt4g6q8BPIfHCE+SJEkWA8vijKg9PavB42UPxPehR0taD0+eMcvMtpLUBXg4ZFDge8+bmdlLFdrqi3tcv4UHNLnKzLaW9B3geOAE4ELKvLTNbGNJl1NIdVnDmxsKXt/lA0gv7iRJkrZhWTTQ7elZDfAnM5sPPC/pRXxPeTdgc3lKSfD93/WBj4AnqhhngPGxTI48RGfJqE/FndGgupd2ObXq3VHJOEN6cSdJkrQVy5yBNrO5kmbQ4Fk9hcae1RvUuL2RZ7WkrfGl4YNxR6ud6xlChXMBx5vZfcULsYz9PtWpJyFHyUu7kYEtGGLqqFdrDEmSJEkbsMztQQet7Vl9AvV5VgMcKKlTOHWtA/wdX04+JpbakbSBpG6t8JxQ3Uu7fJzV6iVJkiTtwDI3gw7aK7EGuEF+EHe8OtrM5kiaiedsflI+ZZ2Je4MDrCFpEzN7eiGfdTqwTXiAL4d/lByN56e+WdLe+H71MOCSCvXqZuors+h9yt0LOcymzMjMWEmSLMNksozFiKRrgTvLHchUI+lFrWt19jkDGGBmdWW8insWKolGlzXXtzUPv6Clt1UlDXSSJMsCymQZC08rSrN6AkOak2YV7m1yLX73SpooaZykjSQtJ090MSju+5WkM2PMn8K9xUeXxlNo/4D4AEDScEm/iXpnV+qnzV5wkiRJ0oRldYm7pbSmNOtR6pRmmdkj5dckjcKXxp+XtA1wqZntLGkovmQ9DPgSsI2ZfSTpe8BOdc6gNwB2NbN5lfqhghNcUWbVeeWedb/QJEmSpDZpoOujvaVZRLvd8ehfIwte2F0AzGy6pOvxveXtzOyjhXjOkWGcq/ZTTlFm1WXN9XO/JEmSpJVIA10HHUCaVaIT8I6ZVfOw7gO8gzugVaNoRMsTYJTkVM31kyRJkrQxaaDrpyTNOhIPBPIbYGIsVT8OXChpdXyJ+xA8m1QjYmba1czukfQY8I+41ESaJWkfPLzmgmtm9q6klyQdaGYjw+N7czObLGk/YDV85n6XpK3N7J3C/W9IGgO8I2ljPHzox3gUskbU6qfWC+qzVg8mpGNXkiRJq5AGun7aRJoVWutK0qx98CXwRteAIcBlkn6C52j+k6RXgLOAXczsZUm/xUN8Ho4vP/9F0msxhouj3ZfxON7dqzxveT834gk9qtLaMqskSZIlgbZSnKSBrhMzG4UbqhJX4TNQJJ0P9DWzPpJ2wZfCkXQI8JKkacDdZvYDYOvwpP4NcCzwArBntNkZD1hyK+69PRCYBexV1FNL+j/gaWAL3Oivg2esukPSh8ARZnaRpBWBz+MhQ/+DxwH/m5n9OpbsDwC6S5oWeZ+RdCKemOM0SffgWuiPaZyhK0mSJGlj0kAvPO2ddKPocb0ysGPsce+KZ7naHzgG+MDMNpe0Oe7c1hJOwbNn/VfSKi28N0mSJFkEUge98JR7dj9Kg2f3OAqe3RH0o+TZDdU9u/cDPqiz/5El5zM8ucbImKmfD2wa5TsCNwCY2RTcua0lTAFGSDqUWC0oR9JRkiZImjDvg1ktbD5JkiSpRhrohcTM5gIzaPDsHkdjz+4m2SgKNPLsxlNK3oLvO99b5xCKCSzOAEbHMvVeNPbObk769DGN/zso3rsHcAm+CjBRUpMVFzO7wswGmNmAzl171Dn0JEmSpDlyiXvRWKye3TXoAbwSx0PLxjcEjyS2GbB5hXtfB/5H0mrAbHw//F5JnYC1zWy0pIeAr+IOZe9UG0R6cSdJkrQei20GLWmYpGciiMeitjU09nibqzdcDTmWq9XpHUvDSBog6aIWDGUcsCbu2f06vlS9wLMbKHl2TwaerOHZfZek9/EkGsWkGydJekqF8J9V+DXwK0kP445mJS7DncCmACcDT5TfGCsBp+NZvO4Cno1LnYEbJE0FngLOD9lWkiRJshhYbMkyJD0L7G5mL5WVtzgxQ+h5TzSzCc3UG04ziSYk9Y46m7VkDEsy5e+83n+D5uoNGDDAJkyo+U+SJEmSlKEqyTIWyxK3pMtxKdAdkq7Gl2Q/hct+3pD0I+B63LsZ4DgzeyTuPRn4GjAf+AswAXfGGhGSou2Ak/C91xXx/eBvWY0vD0n9gatxh6yHCuWDcMO/p6TTgM/iM+QNgO8B2wK748vJe0WEsf740nZ3XOo0NHTRY/BZ6U7AKsDXzWycpE2Ba4BP4CsY+0e869lm1j2Cgvw6+jHgF2Z2U4zttOhjM9xJ7dDy54zZ9iV4Yo4PgG+a2bPxsfIWLs16Mpa0i+fXA5cDXXHp15Fm9nY8xyO4XOsOPMBJkiRJ0taY2WL54Q5Vq8fxabiBWTHOuwIrxPH6wIQ43h03Dl3jfNX4OwZPoUixPI6vx40nwHA88Ef5WKYAA+P4HFzyBDAIn02XxvgQrn3uixu73ePabbhD1/Ixvp5RPhi4ujDG8+L4y8Bf4/hiYEgcf6LwDmbH3/2BB/Al5jWAf+EfCYNwTfSnccP+KLB9hWcbBawfx9vguufSu7gL6FzlvPhOTgcuKDzHpTX+XY/CP5om9OrVy5IkSZKWUbJ55b/2dBK7w8w+jOPlgd9K6odLkEqxrXcFrjGzDwDMrElYymCnmGl3BVYFpuNJI5ogqQewipk9GEXX4x8ClfiL+Sx5Km4wSx7WU/HZ/4b4bPaBSCrRGXitcP+t8Xdi1Ac3rD+W9GngVjN7vqzP7YE/mnt5vy7pQVyy9S7whJn9O55jUrRZXAFoLslFUZq14LzCO7kWGFmod1PFt0PjZBkDBgzIZBlJkiStRHsa6KJM6Lu4N3FffHY4J8pFMzKhCJt5KT6jfjmWpsuTQDS6pbk2C/wXwMzmS5obXzrgy+3LRVvTzWy7WvfjHx3LRVt/CA/vPYD7JH3DzP5WNr6a4ylvs0BzSS7eb+a8GvXWS5IkSVqJjqKD7gG8Zmbz8f3mkify/cCRkroCRNQtaCxBKhnjN2IGWdNr29wTeZak7aNoyCKM++9AT0nbxfiWjz3mqkhaB3jRzC7C93TLpU9jgcGSOkvqiQcbaeJ9XQkzexcPLXpg9CVJfeu4bxbwtqQdouhruEd5kiRJ0k50FAN9KXB46IA3IGZsZnYvbsQmxJLuiVF/OHB5lP0XuBJfdr4dGF9Hf0cAl0h6FPiwucrVMM+5fABwtqTJwCR8ibkWg4FpMfaNgOtKF+QZrJ7F94MnA38DTjaz/2vBsIYAX4/xTAf2rvO+w4FzQpLVD9+HTpIkSdqJxSazSpqnlixsYeRoNfpJmVWSJEkHoV1lVh0RSd2AP+Fe0Z3xcJlv4BKvfaPOF4BjzGw/eQaqS3DHtbeBH+FyqF7ACWZ2h6ShuHd3Z9x57DzcU/tr+Ez/y2b2ViUpFO7c9hVgoDzF4/54Eo2SxOlv0f4G4bi2Mj7TXt882EjpuXricqleUXSCmT0ce/NFadtzZec/xKVnPYGZeEasf5XLs/AEIUmSJEkbs8waaOBLwKtmtgcs8O5+F1/67mlmM/Gl8Guifjc8+cUPJN0G/AL4Ap6G8Vp8KR7cMG+B743/A/iBmW0hT0l5GHAB7vV8tLn+eRtcxrSzyjJYhSf2KmY2MM57485ltwMHA7cUjXNwIR716yFJvYD7gI3jWn9cmvVhGOzi+Z3AdWZ2raQj8Sxd+8R9CzJnlb9ESUfhUit69epVfjlJkiRZSDrKHnR7MBXYVdLZknYws1nhpX09cKg8veJ2eHAU8JzKRZnVg2EcS5KrEqPN7L0w8LNokHtNBXqXSaEmAb/Ddc7VKEqcriJyTdP446HIrrhkbRL+0bCyPOMWNJa2lZ9vB/whjq/H5V4lyuVZC7BCsoyePXvWeIwkSZKkJSyzM2gzey6igH0Zj2N9v5mdjhu9O3Gp18jCnmu5zKoowSq+x6IUan7hvCTNak4KVc4CiVMsVfeWNBAPMDKtQv1OwHZlhrg0G2+JzKronJAyqyRJksXMMjuDlifb+MDMbgDOBbYEMLNXgVeBn+De4q1KQQp1lKSby6RQlTJYHVF2fh3wRyrPnsGlaceVTiL4Sz08gi+bg3uCP1SjbpIkSdLGLLMGGugDPBFLwYA0z3gAACAASURBVD/G95RLjABeNrOn26jvIcB+eFjTohSqUgarcgM9AvgkbqQrMQwYIGmKpKeBo8srlM34S+fDgCNCZvU14DvN3ZckSZK0HR3WQEs6LIzM5EjkgKTPSBoV5aPCCaqUVvIiSY9IelGFFJOSTpY0Ndo5K8q+iRtkwxNDDASelzRDngd5e+BaSS9H8JF1gYckTZQ0DrjRzM4t9WGe5OI0YBdgE0nPS/qmmfUG3pR0Dq7hHihpsHlGr6PxZepNgH9JuhX4KR729AEzewF4DOgiaZKkEeF5fjeu3X5I0uAKr65H/OYCb+JOaeD75J+SNBo4u8L5KnioVHBnufcK920pDznaxGgnSZIkbUSlAN3t/QM2xaN0lZJrlJJk3AkcHsdHArdbQ+KHkfgHxybAP6x2so3VCn39Ajg+jv8c/Y7FZ7lXWY0EFGVjPg0PLrIisDrwMi5jqpb8ojcNSTqGAi/ihnUF4J/A2lZIohHH9+COZxvEeY8K48hkGUmSJEsQVEmWUfcMWtKKkjast/4isjNws5m9AY2SZNTyNL7dzOabL0uvEWXVkm1sJmmcPAnGEPyDANxjerSZ7YhHCLuphV7XfzazD2Pco4GtKSS/MLPX8RCaW1W4d5S5J/kc4GngMxXqnIBrkr9e8jwvXqxjrC1JlrFjoV7NZBmWXtxJkiStTl0GWtJeeBjLe+O8X2h224p6E1oU6xS9p1X4W6md4XhAkj7Az2mI530HsLs85nd/PNTmAq/rwm/jCm2Wj6d0Xiv5RZHmEmFgZs/FuKbinuenllVpbqyZLCNJkmQJod4Z9Gn4bPAdADMrpTpsK0YBB0laDRolyWipp3G1ZBsrAa9JWp5Csgwzm40nprgQDxgyz1qWgGJvSSvEuAfhccEXOvlFMDfGWdXzvDD+TJaRJEmylFCvV+7HZjZLqncyuGiY2XRJZwIPSpoHPIXv0w4DrpZ0EhGOspl27g2Z0QRJH+F7uD/CnbEex/d6p9JY2nQTvp89qFA2BLhMHoJzedzbenKFLp/Anbh6AWeY2avyqGPbRX0jkl9EVLBqbIDPlMfg4TWnSpqIS6zOkTQfdwI7psK99Y61nMPxBCRd8f3wmu82SZIkaVvqSpYh6ff4rPYU3OlpGLC8mTWR8CyrhBf3bCt4d7dSuzPwXNdvtHK7nYv70eXn9d5XJJNlJEmStBxVSZZR7xL38bgj1X9xJ61ZuMPSUkNby7okjY+yWyR1ldSjIOsiykqyruGSDpA0DPcEHy1ptKSvy2N6U2j3NxWeZTdJj0p6UtLIcB4j+jtV0kPAgRXOD4mxT5N0dqG92ZJOl/Q4vhqQJEmStDWVXLutsYymM/DX5uotyT/aV9a1UxwPpkHWNRw4II5nFMbVDddtLx/njwB9yp5ldXzfu1uc/wA4tdDWyYW6C87xD4F/4dmslsMd5PaJawYcVOXdpcwqSZJkEWBhZVbmy5kfhBRnaaU9ZV2lYCMHU0POFO29jxvOPSVthBvqqWXVtsU/Gh4OqdXhNJZslfdROt8Kz9Y10zz++AgapFbzgFuqjCllVkmSJG1AvU5ic3BHpQdonLxhWJuMavGzOGRd+5jZZHlO50FRfgculyrKuprjKtzR7Vkqx+MWHonskCr3V5Na1fIAnGN17E8nSZIkrUe9e9B3457PY4GJhd/SQoeRdVVos1ECDTN7HFgb+CqV43E/Bnxe0noxhq6SNmhm3OBe7QMlrS6pM3AIKbVKkiRpN+qaQZvZtW09kPbE2kbWNQ94BfgSLZd1FbkC+Iuk18xspyj7E9DPzN6uMIaZMUv/o6QuUfwT4Llmxv6apB/iEdAE3GNmf651T5IkSdJ21CuzeokKS7dmtk5bDGppIHTOd5nZZm3Q9l3A+WY2ahHaaLHMSi6El5nNr3Q9ZVZJkiQtZ1FlVgNwJ6KtgB2Ai4AbWm94C89CyKMuC8nSi5IGSrpa0jOShhfanC3pvJApjZJHAKsol4ryNSTdFuWTJX0OOAtYV56J6hxJgySNkeeAflaenUpxf39JD8qzZd0nac0oHybp6XiOGyWtIull3FntPHlayvL80Ug6VNIT0ffvYsm6iVyqwvn3QmI1TdIJcU/veD+X4kFT1m6Tf8gkSZKkMZVcu+v5AQ8t7L2t9WPh5FE34ku4e+NpFfvgHyoT8WVj8NWCIXF8KvBbqy2Xugk4IY4741mpehPZqqJ8EK4f/3T09yhuaJfH97p7WoPc6uo4fhXoEserFJ7t83HcHViu7J1sHHVKUqxLgcMKz3VQoe6CcxpifHeLdqcDW8RzzAe2rfJvkDKrJEmSRYAqMqu69qAlFWM+d8Jn1E1mbu1ALXnUfnF8PfDrwj13mpmF5Ol1C5mSpOm4MZqEG6SS/OgG4NY43kzSL/Dcyd2B+wrjOCzGMA+YJemTFcb7hJn9O/orxTN/B9gMeCAm1J2B16L+FGCEpNuB26PsYeA3kkYAt5baK7ALbmzHR3srAv+Ja+VyqeL59sBt5lIu5Pmpd8A9zf9pZo9VeB7M7Ap8n5wBAwbU4wmfJEmS1EG9MqvzCscfAy8BB7X+cFrMosij5tNYKjWf6u+jdP9wKsul6qVSxioB082sUoSuPXAt8leAn0ra1MzOknQ38GXgMUm7mtmzhXsEXGtmP6zQXrlcqnheS2aV2aySJEkWM/XuQX/dzHaK3xfM7Cjgo7YcWJ20ljyqnE54PmhwOVPp/nK5VHdJX41xHCNpqKRLJK1MmTyqBn8HekraLp5heUmbykOArm1mo4GTiVm7pHXNbKqZnY0vLW9U1t4o4ABJ/xPtrSqpUm7pcsYC+4QsqxuwLzCujvuSJEmSNqDeGfTNlKU2jLL+rTuclmGtJI+qwPvApvIMUrNoiPZVLpdaHzfgX8eXebfEc0vfYGaPSnpY0jTgL7iWvNIzfCSP5X2RPFrbcsAFuCzqhigT7rX9jqQzJO2Ez8CfjraL7T0tz2R1fxj5ucC3Y8wASFrOPFpY8fzJcJQrpcK8ysyeUiHrVnOe3lNfmVXtUpIkSdJCahpoeTjJTYEekvYrXFoZN0TtjrlGe4FOO2Z/lwCr4TPhK8zsX/KsUH8Avi/pFNy56RVJLwDnmNlQOefg+7b7AL8ws5sK5bsDs4HLovwx3CnrvhjDbfhy9M8krYvv6X41xjUbuFDSZOBDYG8zez08xH8afX4EHGtmD0saiO9zz8OX2C8J7+6++DbDcsClZlZcNkdSf+DYuO91YKi5xnkM/hHwIHCHpL3KzifheaANz2N9WTQ5Bv/YeQj4Le5klyRJkrQxzc2gNwT2xJdX9yqUvwd8s60GtYh8CXjVzPYAUOMY4i+b2XbyjFDDgc/jHxrTgctxx7J+uAHdFXe0Ggt8Lsr74skoSuWnACea2Z7R19CotwW+3/x3SReb2cu4d/RjZvZjSb/G398v8Chi55vZQ3I52H240T8R+HYY6+54uNWjgPvM7MyQTnUtPngsvV+MG/+ZkgYDZ+Ke7OCe4AOj7l6lc0krAM8Du5jZc5Kuw3NNXxD3zTGzYhzyYp9HxbjovHLG4k6SJGktahpo80hSf5a0nZk9upjGtKhMBc6Vp0u8y8yK+6h3FOp0N7P3gPckzZG0Cu7J/Ecz+wJAzC63KpTPA14vlL9bof9RZjYr7n8aT1TxMj47vivqTAS+EMe7ApuExzXAyqFtbuKtLWk8PptdHpeOTSrre0Oqe4RD9UQZGwIvmVkp2ti1+LL4BVXuW0DRi7vLmuunF3eSJEkrUe8e9FOSvo0vdy9Y2jazI6vf0j7EDLA/7uX8K0n3m9npcbk5D+5qnsy1PJzLqeSpDTA39G7l5Z2A7czsw7J2Knlrj5W0I+7dfb2kc8zsurJxVvMIh4VLlFHpviRJkqSNqdeL+3rgf4Ev4gkUPo0vc3c4JH0K+MDMbgDOpalzWy3GAoMldY694R1xp6lq5fV6atfifuC4wvj7xd8m3trhjf0fM7sS+H2FZ6voEV7HGJ4FeisSbOB70S1OlNFnraU5I2mSJMnipd4Z9HpmdqCkvc3sWkl/oCFIR0ejD3COpPm4B/MxLbj3NjzIyWTcWepkM/s/SdXK3wQ+Dsev4UCT5BV1MAx3AJuC/3uMBY4GTqjgrX0wcJKkubiz2mHFhmp4hE+vNQAzmyPpCGCkpOVwJ7HLF+JZkiRJklai3mQZT5jZ1uEYdSzwf3hUrEyW0co0J2VqpT4qyaw+rnVPPfUyWUaSJEnLUZVkGfXOoK+Qh678Ke5o1R2PUZ20AHnIzrXxffwLw8GqJMH6Db6F8P3QHg8DPoHrro81s3mSLsOd01bEQ5z+rEIf6+Iys57AB8A3zezZ0Di/hXuYPykP7lI8vx6fNXcFXgCONLO3Q571CO7xfgeNo8olSZIkbURdM+ikdZC0qpm9JWlFfBl5oJm9KcmAwWb2J0kb47HD9zOzufIsUo+Z2XWF+zvjEcOGmdmUsj5GAUeb2fOStgF+ZWY7h4FeHZdgzatwPgVP/vGgpNOBlc3shDDQT5vZsVWeaYHMqlevXv3/+c9/VqqWJEmSVGGRZtCS1gB+CXzKzHaXtAnuefz7Vh7n0s4wSfvG8dp4JLI3aZy0olayi4PCIC4HrAlsgifUACD00p/D95JLxV0K/Y8sWz4fGca5B66JLjmGXQuMLNSrS2aVyTKSJElaj3qXuIcD1wA/jvPn8P9pp4GuE0mDcM3zdmb2QcxMS5K18qQVTZJdSPosHrxkq1h6Hk7TaG6dgHfMrF+VYVSTWTVHyqySJEkWM/XKrFY3sz/hemHCUahNHZmWQnoAb4dx3gjYtkq9askuVsYN5axY0di9/EYzexd4SdKBca8k9W1uYBFY5W1JO0TRQsmskiRJktajXgP9fjgVGYCkbfEkEss8ko6WdFjzNbkXWC72es8AmuRXlvQjM3saKCW7mAI8AKxpZpPxZCDTgavxSGOVGAJ8PaRf04G963yUw3F52hQ8XOnpzdRPkiRJ2pB6ZVZb4jGeNwOm4R7CB5Q7KC1r1CtPakF7s82sewvvabEsK2VWSZIkHYeFchKT1MvM/hWpCAfiMZsF/N3M5rbRWBcbIWe6F5cybYHvrR8Wy9D9celTd+ANGmeFWiA7irjZs83s3Lj2FO7k1RMPJPJDPHjKTWb2k+j3UMpkVHhSixXlWaWmm9mQSvXCqauRLItCvuuUWSVJkiwdNLfEfXvh+CYzm25m05YG41xgQzwl5eZ48otj1ZAV6gAz648vKZ9ZuGcVMxtoZpWM1UdmtiNu7P6MJ53YDBgqabWQUQ0GPh/OXPOAIWZ2CvChmfUL41yxXvTRDZhmZtuY2UONu+cKXC7VH3cqu7RwbQNgVzP7foXz64AfxHuYChQ11lWfV9JRkiZImjBz5swKryNJkiRZGJrz4i4mUVhao4a9bGal/dwb8BnrvbQsK1SRYsas6Wb2GoCkF3Fp1fZUl1EVqSW3KsqyFpAyqyRJkqWH5gy0VTlemih/LqPlWaGK1JMxq4mMqgK16s2xyvvOKbNKkiRZSmhuibuvpHclvQdsHsfvSnpPUqVcyEsivRTZn4BD8P3c5rJCdZdUMbJWHYwCvirpkGi7JKMCmBvL66V6leRWVUmZVZIkydJDzRm0mXVeXANpR54BDpf0O+B54LI6skKthDt2XVqpwVqY2dPyPM8XSfohnnHr28A/8aXiKZKejH3oktyqU9Q7LurVYghwWdy7PHAjnoWrnPIc0EcAl0rqCrwY501vquE1PvWVWfQ+5e5mhtcyZpy1R6u2lyRJsqRQrw56aWa+mR1tZpub2f5m9gGAmU0ysx3NrK+ZbRo5mDGzQbgRXDc8rruFB/dJuPPW1ZJ+bmZjgN9L+qt8Q/gQ4A+SegF7lvrG425/SdKJZvYDM9sYX7nojXtud8Fn9Z2AV6OfZyRNkfTz8ocxs5dwD+8PgDlAH0ndzWwocK6kUyU9hKevLJ5vAFyIG+71gFMKzztG0umSHsfTbiZJkiRtTBroheMU4IXwuD5J0m54XO2t8SAf/SXtaGa34ak5vw1cCfzMzP6FZwK7Ke6v5XAG7mV+nZltEcdN+ilWlrQ6HuhkVzPbEpgAfK9QZY6ZbW9mNxbP8TzUZwM7R9tbSdon6lT1Gi96cc/7IGPXJEmStBb1xuJeKjGzGbi39qKyW/yeivPuuCEdCxyPB3d5zMz+uBBt/9PMSlHHavVTYls8icbD4cn9CeDRwvXyD4LS+VbAGDObCSBpBLAjLrWr6DUOjb24u6y5/tLqSJgkSbLYWaYNdCsiPK3j7ypcWwtfyl5DUiczm1+hzsc0Xs0oJsEoelDX6qdY5wEzO6TK9Wqe3OV70kWqeY0nSZIkbUQa6IXjPdxRrMR9wBmSRpjZbElr4U5db+FZwL6KRxX7HnBuhftnEPvSEVb1s1X6rdiPmRV11I8Bl0haz8z+EU5fnzaz55p5pseBC2OJ/G18z/ziZu5pRJ+1ejAhnbqSJElahTTQC4GZvSnpYUnTgL/EPvTGwKOxrDwbOBQ4GhhnZuPCoWx8eHCPBk6R9A886cj2wPclvQrcjYccrdTv/VX6+U+hzkxJQ4E/SioFKflJtTYL970WXuWj8dn0PWb255a8l3Iv7vTATpIkWXjqSpaR1E94bKvKUnZ53UHAiWa2Z3N1W5NFSJZRMzFHlzXXtzUPv2DBeRroJEmS5qmWLCO9uFsBSb0lPSPpUuBJYG1Jl4V38/SiHErSlyQ9G9Km/QrlQyX9No6Hhw67dG12/F1T0lhJkyRNKwQWKY6lv6QHJU2UdJ+kNaN8jKRfSnoQ+E6F810kPSVpqqSrS7NvSTMKUqwD2+L9JUmSJE3JJe7WY0PgCDM7FkDSj83sLUmdgVGSNseXma/EpUz/oHZM70p8FbjPzM6MdrsWL6ohycfesdQ9GE/ycWRUWcXMBkbdvUrnklbAg7TsYmbPSboOOAYPzgINUqwmSDoKOAqg88o9W/g4SZIkSTVyBt16FOVQAAdJehKXRG2KS582Al4ys+fN9xZuaGEf44EjJJ0G9DGz98qub0hDko9J+N7zpwvXq0msNoxxlfapr8UlVtXuW4CZXWFmA8xsQOeuPVr0MEmSJEl1cgbdeiyQL0n6LJ7qcavIqTycBulUPZv+C2RXsaf9CQAzGxuBSfYArpd0jpldV7ivpUk+6pFYVbovSZIkaWOWyhm0pI1in/YpSesuYlv9JH25jqrdJN0VxyvjRm2WpDWA3aP8WWAbSXvH+eVU/kiagaeaBNgbj6mNPFnGfyLs6O+BLcvuay7JR+mZTgc+WSh6Fugtab04X6hkGX3W6sGMs/ZY8EuSJEkWnqXSQAP7AH82sy3M7IVSoZyWPnM/oB4DvQAzm4wvbU8HrgYejvI5+D70heF0dR0+Wy7nSmCgpCeAbWiYwQ4CJkl6Ctgfj51d7Pcj4ADgbEmTgUl4fujy8Z2Ka51L53Pw5BgjJU3FA6tc3pJnTpIkSVqXNjHQ4dX8rKSrwtt4hKRdQzv8vKSto1638BgeH7PdvQv3j5P0ZPw+F+WDwvv45mh/RCwBF/v+MnAC8A1Jo1voYb2VpEckTZb0hDyT1enA4JiRD5a0ddR5Kv5uGCFDG2V/MrOhkfjiAOAj4HuSbgI+BA4Ip6uDgaHyxBinACvItdW/wfeP58b9O0ezNwNP4Ea9O7B5jHuopFsl3QuMxMOK9o3rn482VwN2iPrDgd+a2YSS9zbuEPYUMMDMjgT+Hu/mLWC0pI1a+J9BkiRJsiiYWav/gN64EemDfwRMxGeSwpdsb496vwQOjeNV8NllN9w7eYUoXx+YEMeD8MAen452HwW2r9D/abi+uDSW+cC2heurxt/OwBjckH0CT7O4VVxbGV9+HoobM4rlcbwrcEthbHdVGMv3gKvjePN4LwPifAaweiu8r6Ex9h74Xvc/gbXxZfIHCmNZJf4Oxw3/CsDLwAZRfh1wQmFsx8fxscBVzf279+/f35IkSZKWUbJx5b+2XOJ+ycymmgfsmA6MioFMxQ0SeOKHU8LjeEwYjF74nuuVsdw6EveALvGEmf072p1UaKsW9XhYbwi8ZmbjAczsXascvKMHvhQ8DTg/7q/FjoS3tplNAaZUqbco74uoP8t8ufpp4DO40V5H0sWSvgS8W9Znc97bt8bfiVR5zypks5o5c2b1t5AkSZK0iLb04v5v4Xh+4Xx+oV8B+5vZ34s3hozodaAvPqOcU6XdedT3DPV4WIv6PKzPAEab2b6xND2mjnvqaXdR3tc2VHgv8Xx9gS/iKS8PokETXWqvnjFVfc9WyGY1YMCADEuXJEnSSrS3k9h9wPGlfWRJW0R5D3w2Ox/3KO7cin3W8rD+lKStYiwrSVqOpoktegCvxPHQOvobCwyJNjcj9o0XkmrvqyLyxBedzOwW4Kc09fpuFe/tJEmSpPVpbwN9Br6cPSWWjM+I8kuBwyU9BmxAK+pwrbqH9UfAYODi8IB+AJ9ZjwY2KTmJAb8GfiXpYer7cLgM6C5pCnAy7uS1sFR7X9VYCxgTS+LDgR8WrvXEQ42m93aSJEkHJJNlLKOoRqIO1Zk8o5wBAwbYhAkTWmN4SZIkywxa1pNltKf0K+oNk/S0pCmSbpTUKfrtGdc7SfqHpNXlyTIuC5nYi5IGxpieiT3zUpuzJZ0tT4zx15CAjYl7vhJ1Oks6J55niqRvxe1nATvEysB3Q6o1UtKdwP2SrldDQBXiub7SNv86SZIkSRMquXYvjT/aX/r1KtCl1G78/RkNsqbdaJBsDQduLIzt3bJx94t6Buwex7cB9+NL4H2BSVF+FPCTOO4CTAA+S5ksDN9P/zcNErSBhXfSA3iJkJeVPddR0eaEXr16WZIkSdIyaAeZVUfkJWs/6dcUYISkQ2mIHnY1cFgcHwlcU6h/Z2Fsr5eNu9T+R8C9cTwVeNDM5lZ4nsPieR7HA5asX+X9PGBmbwGY2YPAepL+BzgE/3hosuxthWQZPXtmNqskSZLWYllLltGe0q89cI3xV4CfStrUzF6W9LqknfGQnkMqtFkcZ/lY54YRb1TPzOaHB3rpeY43s/vKnmdQhTGWO+NdH2M6mMbyrCRJkqSNWdZm0PXQ6tIvefzvtc1sNO7JvQoeqhPgKjyQyZ/MbF7rPEIj7gOOkeeKRtIGkrrRVD5WieF42FTMbHobjC1JkiSpQhroprSF9KszcEMsjz8FnG9m78S1O3BjfU21mxeRq/DIYq9Leh74HT4DnwJ8LI87/t2o+1lJC5buzex14Jk2HFuSJElShZRZtTOSBuAGe4cOMJbhuOPYzXHeFd/P3tLMZjV3f8qskiRJWk6Hlll1AAnUeiFTmhz3ryvnnBjPVHmQkpptqmk2rJVqjO0mSdcAtwA/DGnV/jVkUZXe17VR5+YwpiiyU8WYr5bUJcrHxMdASZ51ZozzMUlrxLi+ApwT0quv4V7d3YBxkm5s1X/0JEmSpDaVXLsX94/2l0A9DuwbxytEe/vj0cQ6A2sA/wLWrNYm1bNhVRvbvsC1cfwJPKvUilSRRVV4XwZ8Ps6vxuOL18pONYaGLFoG7BXHvy70NxxPhVlVGlbh3aXMKkmSZBFgCZBZvWTtIIGStBKwlpndBmBmc8zsA9zo/tHM5pnvxT4IbFWjzWrZsKqN7S/AzjHD3R0Ya2YfUr8s6mUzeziOb4jxNpedqsRHwF1xXDVTFZWlYY2wlFklSZK0CR1JZtVeEqhqGZ1qZXqq1Ga1bFjfrTQ2M5sjaQyeaWow8MdCv01kURUo78uaGXORojyrVkawStKwFocATZIkSVpOR5pB10OrS6DM7F3g35L2iTa7xH7uWGBw7An3xA1VrUQX1bJh1RrbjXiyih3i2UrPWEkWVU4vSdvF8SHAQ7QwO1XsSR8Zx4Pw7YKV4ryWNCxJkiRpY5Y0A91W2a++BgyTZ5x6BPhfPHTmFGAy8DfgZDP7v2oNWPVsWLXGdj9u+P8a90ODLOrJeMaSLKqcZ6LdKcCqwGVmNocWZKcyswn4/jX43vrrwEmSnsKX1atJw5IkSZI2pkMscZvZDGCzwvnQStdij7aJV7OZPU/jPMs/jPIx+F51qd5xJQ9ofMa5LW6Ar8EdslYEvmFmL8asdTXgQ/w9lZbNZwA9JD0Z58eZ2SMxAz0N93zeDJgGvF8+Nkm/l/RXPN3jPGBr4EVJ5+B70Qb8wsx+VGjz9/Jc0hOBQ6OpLtGuAWsDnSX1jvoGzAWuMrP/SroJ+HUYZICbJe0PvInH4D4OeCzGMxMYhocQ3cDM5kpaGf8oOs88lGiSJEnSxnQIA90OrAcciHsgjwe+ijtZfQX4EbAP8GPgb2Z2pKRVgCfCsP4H+ELsIa+P7x2X9GtbAJvi3s8PA5/HPwSKjADOMrPbJK2Ar2LsB/TD96lXB8ZLGlujzddxo3ywmY0PA/phjbHdiM/u75H0CWAX4Bg8vChmNkPS5cBsMzsXXJaF70Hfjof6vKWScZZ0VLxHevXqVcerT5IkSephSVvibi2WdI/xFYDxreAxXour8OVy4m/FaGLpxZ0kSdI2LKsz6PQYbwYzezi2AwYCnc1sWnP3JEmSJK3HsjqDrodlwWO8SKXkGdfhxjxjcSdJkixm0kBXZ0nzGP86cHONsVXyGC9yJ7BvhPksxQUfAXySOmbcSZIkSeuSyTKWEmIJ+8SCp3ZrtHkAsLeZfa2e+pksI0mSpOWoIyfLqJdCkoglJanGg5L+JOk5SWdJGiJPojFV0rpRb7iky2Ncz0nas9ZY49rJ0cbkaPcA3Ft7RMyAV5Q0Q9LP496pkjZq5t1sGmObJE/AcR1wNrB29DOt9GxJkiTJYqBSgO6O+mPJS6rxThx3AV4Bfh73fge4wBoSVNwb/a6P66hLbVca6+740njXOF/VypJhxPkMPGQowLG4JrrWu7kYGGINyTtWjGe7stBmjwrvJJNlJEmSLAIsqnraFwAAEvxJREFUAcky6uUlW3IkUuPN7DUz+y/wAr4PTNlYAf5kZvPNg5q8CGxUY6y7AtdE35jZWzXe1a3xt5gQo9q7eRT4kaQfAJ8xl2FNBXaVdLakHaxCTmhLmVWSJEmbsCTKrJZUiVS1sULlxBcVJVNUl1jV6r/4PBXfDfCMpMfx4CT3SfqGmf1NUn/gy8CvJN1vZqfX2XeSJEmyCCyJM+h66MgSqUocKKlT7EuvA/y9xljvB46MvpG0apRXkklVouK7kbQO8KKZXQTcAWwu6VPAB2Z2A3AusGULnytJkiRZSJZWA91hJVJV+Du+NP4X4GjzpBdNxirpBPyD4A5ggqR5wInRxnDg8pKTWI2+qr2bwcC0WPreCNdA98FDnE7CQ5/+ooXPlSRJkiwkKbNqZyQNB+4ys5vrqDsDdwR7I85nm1mHSQGZMqskSZKWo6VBZlUJtb/0apikp0OadGOUnSbpWkn3h9xpP0m/DrnTvWrI9bwLsBdwXoytS6k8xji1VC5pGPApYLSk0YX+zwwZ1GOS1oiy4ZIukvSIpBdDhlWqf1K8gymSfl54N3eXy6nkEq7Ss53b+v96SZIkSTWWeAMdrAdciKdf3IiG7FQn4tmpoCE71VbATsA58pSSpQxQW+LLvBcV2t0COAH3oF4HzyRVzinAFma2OXB0oXxd3OFqb+AGYLSZ9cGzTu0hz2Q1HNjOzD6DO3EdUygfHPWXA46JveFXgZ3MbKfooxvwmJn1xZe+v1nof814B3sCZwFI2g2XbG2NZ8/qL2lH4EvAq2bW18w2A+6Nve19gU3j2Soub0s6StIESRNmzpxZqUqSJEmyECwtBrpdpFfBFDxAyKG4RrvEX8zTM07FHbzujfLSmDaMcT8X5dfiDmbVyivxEXBXHBelVOCa8Plm9jSuzy69g92Ap4An8Y+Z9aksp3oX9xy/StJ+wAeVBpAyqyRJkrZhSZRZVaK9pFfgs+Qd8VzSP5W0afFeM5svaa41bPaXxrQwsq1yiu2Wj684dhX+/srMftek0wpyqtge2AXPB30csHMLxpYkSZIsAkvLDLoeWl16JakTsLaZjQZOxiNz/X979x5sVXnecfz7i6jEGyBgtVqC9RIvFDEiEcYxJnFSNVGxmtFUoyTGNJnRaRK1SWdyQUlSNWl0qnEscSi1zai1eMMrREWiiHdAvFCNEKMypkaDGKkKPv3jfTdnsdnnnLUPZ++9jvw+M3vYe613rfXsxYH3rLWe933KJm09C4yWtGf+/EVSJnd3y6H8UKru3EUaorVdjn9XSTs1Gk6V2wyJiNtJt/nHbcJxzcysSR+UK+gypgGXkoYXiTQV5udIw5lmSfo8cC/NDb3aAvhPSUNIV6eXRMQfG+SSbSRSjeYvAdcrlYp8BLgyIt5ptDxvNh24Q9LKwnPo0iJijqR9gQdzjG8Bp5Ke4f9E0vvAe8DXSb8I3JyfiYs0cYqZmbWJh1m1iKTRpOfO9wOHkMZJ/xtwPrATad7rh3Oi2mWkMceDgKkRcXPe/j9IiWAAZ0XEAkmHA1OB14AxpGfPp0bdX6SkM0nzZG8FPA98MSLezsO6XiclwD1O+gXl58BI0nPmMyPiWUnHAN/N2/8hx/tqT9/Zw6zMzJr3gR1mVXGdzC6/ISIOzhnez5DqRdfsDRwREeeQrsrPjoiDclxX5Db3A4dExIHAtaRb+BtxFreZWWtsTre4O2F5RDwJIGl9dnnOGB+d23wGOFZSbUawWnb5K8DlksaREsD2Luz34Yh4Ke+3ll1+f92xx0j6IV3Pxe8qrLs+Itbl58yTSLfTa+u2zn/uBlwnaRfSVfTyRl8wIqaTOnnGjx/v2zFmZv3EHXRrdTK7fCYwOSIWS5pCKn9ZU3vO/iHgjxHRKAHsMuBnEXFL4ba6mZm1iW9xd16/Z5dn2wMr86xlpzRqkAuALM8Jcig5oHD8l/P705s8tpmZbSJ30J1XurBHHg51fsn9fg94CJhLGrq1AUmTJe1H6rzPkLSYNMnLcbnJVNKt71+TEtLMzKyNnMU9QEgaFBFre29Zen8zKVmko2wMzuI2M2ues7ipRGGNeZIuVSpisbTE8aZIul7SbGBOPv7SwrqbJM2WtFzSWZK+lbdfqFwnWtIeSgU6Hsux75PjPpaUMb4ot9moXd5+pqSfKRXouKjVf0dmZpZsjkliewKfJ40RfoSuoU/HkoY+TaZr6NOXJQ0l1UT+FV1Dn/5P0l7ANUDtt54Dgf1J2dcPkIY+1WdWA2wbEZOUilTMII1l7u54ABOBsRHxeh4bXTQmH3cwaazztyPiQEmXAKeRJmaZTqox/ZykjwNXRMSnJN1C4Qpa0t317eia2rM2LGtd/ZeR9NV8Lhk1alR359zMzJq0OXbQnRz6BKlTJyLmS9ohd8jdHQ9gbkS83s13uTciVgOrJa0CZuflTwJjexlGtV6Jdtc36pzz9/AwKzOzFtgcO+hODn0CqO/EoofjfZyepx7t7bv0NIyqqLd2zUx/amZm/WCzegbdhFYNfYI0KxiSDgVW5dKO3R0PYJ+cvd20PIzq3W6GUa0vvNHLcCszM+sAd9CNlR761Id9vyFpAakARm36ze6OB7Av0KcOOhtB42FU1wLn5aSyPeh+uJWZmXXAZnWLOyJWkBKrap+nNFoXEWuAv4OuoheSrmLjohcrJU2IiHk5A3sGXUUv5ha2rxW92Au4OiL+UWl2roslFYtejK0revEW6ar9l5LWkBLGTpd0H2n6zteUpuJ8m3R7ezhpzPJfA/dIupD0PHtnYCkpGe1W4IKIeCDHu11E/EbSPGABKbntFmBe4Ti7SHogIlY2fdLNzKxPfAVdTn8VvXg6b1PTY9GLnGH9KKmS1DhgLWkKzhNzcYsZwI/ybfKzgJmSTgaGRcQvIuI7wJqIGBcRDWcTqzM0Ij5BKsyx0XFKniszM+sHm9UV9Cboz8zv0YX9ls38rvko6Wp7bn5cvQWwEiAi5uZnyD8nJbH1xXW9Haeeh1mZmbWGO+hyOp35vX53wFMRMXGjFdKHSM+r1wA7Ai812H4tG941GVy3vvZMvdvj1PMwKzOz1vAt7v7Tqszv9dnWwDJgpKSJ+RhbSto/r/smqe7zF4AZSkUyAN4rvH8V2EnScElbA5/r5pg9HcfMzNrAHXT/aVXm90zgynwLfAvgROCinG29CJgkaW/gK8A5EfFrYD7w3bz99BzTL4HngH8mFdF4lQZFNAAi4t1Gx2kybjMz2wQuljHAqYkiGpJWAOMjoiXVqVwsw8yseXKxjP6jahTd+HEeBvX3ko6R9FA+xq8k/VluN1zSnLz8X0nPlmv7eKtwzFsLyy+XNCW/v1DS05KWSPppy06omZltxElifdfpohu1IVFIGgYckjPLvwL8A3AO8APg/oi4QNJnc6ylKFXDOh7YJ+93aDftnMVtZtYC7qD7rtNFN64rvN8NuC5PWrIVsDwvPwz4G4CIuE3SG018vzdJGedXSbqNNMHJRpzFbWbWGr7F3XfNDL0al1+jIuIZUsZ1bejVeFKn2mi/PQ29KiabXQZcHhF/RZoBrTh8qrdOs+HQq/xcewIwi3Q34M5e9mNmZv3IHXRrtbLoRtEQ4OX8/vTC8vmkObaRdBQwrMG2vwX2k7S1pCHAp3P77YAhEXE7abaz3ipimZlZP/It7taaBlxKGuYkYAVp7PEVwKw889e9bHo5x6mkWs4vAwuB3fPy84FrJD0O3Ae8WL9hRPxO0n8BS0jDsJ7Iq7YHbpY0mHQn4JubGKOZmTXBw6ys33iYlZlZ8zzMyszMbABxB21mZlZB7qDNzMwqyB20mZlZBbmDNjMzqyB30GZmZhXkDtrMzKyCPA7a+o2k1cCyTsfRhBFAS0pvtshAincgxQqOt9Ucb88+EhEj6xd6JjHrT8saDbavKkmPOt7WGEixguNtNcfbN77FbWZmVkHuoM3MzCrIHbT1p+mdDqBJjrd1BlKs4HhbzfH2gZPEzMzMKshX0GZmZhXkDtrMzKyC3EFbUyQdKWmZpOclfafBekn6l7x+iaSPdSLOQjy9xbuPpAclvSPp3E7EWBdPb/Geks/rEkkLJB3QiTgL8fQW73E51kWSHpV0aCfiLMTTY7yFdgdLWifpxHbG1yCO3s7v4ZJW5fO7SNL3OxFnIZ5ez2+OeZGkpyTd1+4Y62Lp7fyeVzi3S/PPxI5tCzAi/PKr1AvYAvgN8JfAVsBiYL+6NkcDdwACDgEeqni8OwEHAz8Czh0A53cSMCy/P2oAnN/t6Mp1GQs8W+V4C+3uAW4HTqxyvMDhwK2dirEP8Q4FngZG5c87VTneuvbHAPe0M0ZfQVszJgDPR8QLEfEucC1wXF2b44CrI1kIDJW0S7sDzXqNNyJ+HxGPAO91IsA6ZeJdEBFv5I8Lgd3aHGNRmXjfivy/G7At0Mms1DI/vwBnA7OA37czuAbKxlsVZeL9W+CGiHgR0r+/NsdY1Oz5/QJwTVsiy9xBWzN2BX5X+PxSXtZsm3apUixlNBvvGaS7FZ1SKl5Jx0t6FrgN+HKbYmuk13gl7QocD1zZxri6U/bnYaKkxZLukLR/e0JrqEy8ewPDJM2T9Jik09oW3cZK/3uTtA1wJOkXt7bxVJ/WDDVYVn9FVKZNu1QpljJKxyvpk6QOupPPdEvFGxE3AjdKOgyYBhzR6sC6USbeS4FvR8Q6qVHztioT7+OkeZzfknQ0cBOwV8sja6xMvIOAg4BPAx8GHpS0MCL+p9XBNdDM/w/HAA9ExOstjGcj7qCtGS8Bf1H4vBvwSh/atEuVYimjVLySxgJXAUdFxB/aFFsjTZ3fiJgvaQ9JIyKiE4UTysQ7Hrg2d84jgKMlrY2Im9oT4gZ6jTci3iy8v13SFRU/vy8Br0XEn4A/SZoPHAB0ooNu5uf3ZNp8extwkphf5V+kX+heAHanK6li/7o2n2XDJLGHqxxvoe1UOp8kVub8jgKeByYNkJ+HPelKEvsY8HLtcxXjrWs/k84miZU5vzsXzu8E4MUqn19gX+Du3HYbYCkwpqrx5nZDgNeBbdsdo6+grbSIWCvpLOAuUgbkjIh4StLX8vorSZmvR5M6kbeBL1U5Xkk7A48COwDvS/oGKZPzzW533MF4ge8Dw4Er8lXe2uhQ1Z2S8Z4AnCbpPWANcFLk//UqGm9llIz3RODrktaSzu/JVT6/EfGMpDuBJcD7wFURsbSq8eamxwNzIl31t5Wn+jQzM6sgZ3GbmZlVkDtoMzOzCnIHbWZmVkHuoM3MzCrIHbSZmVkFuYM2s17lKj6LCq/RfdjHZEn79X90IOnPJf13K/bdwzHH5dm7zFrC46DNrIw1ETFuE/cxGbiVVM2oFEmDImJtb+0i4hXSmOC2kDQIGEeaeez2dh3XNi++gjazPpF0kKT7ctGDu2pVyySdKemRXMBhlqRtJE0CjgV+kq/A98gFE8bnbUZIWpHfT5F0vaTZwBxJ20qakff5hKSNKg5JGi1paWH7myTNlrRc0lmSvpW3XVir55uPf6lSXe2lkibk5Tvm7Zfk9mPz8qmSpkuaA1wNXACclL/PSZIm5H09kf/8aCGeGyTdKek5SRcX4j5S0uP5XN2dl/X6fW0z0Ykp1vzyy6+B9QLWAYvy60ZgS2ABMDKvP4k0ExPA8MJ2PwTOzu9nUpg6E5gHjM/vRwAr8vsppHmSd8yffwycmt8PJc3bvG1dfKOBpYXtnwe2B0YCq4Cv5XWXAN8oHP8X+f1hhe0vA36Q338KWJTfTwUeAz5cOM7lhRh2AAbl90cAswrtXiBNGTkY+C1pDuiRpGpKu+d2pb+vX5vHy7e4zayMDW5xSxoDjAHm5ilHtwBW5tVjJP2Q1LlsR5pKsVlzo6ty0GeAYyWdmz8PJs1J/kwP298bEauB1ZJWAbPz8ieBsYV218D6Qh47SBpKqhB2Ql5+j6Thkobk9rdExJpujjkE+HdJe5GqIm1ZWHd3RKwCkPQ08BFgGDA/IpbnY23K97UPIHfQZtYXAp6KiIkN1s0EJkfEYklTgMO72cdauh6zDa5bV5z3WMAJEbGsifjeKbx/v/D5fTb8f69+ruOg5zKEPc3HPI30i8HxOYluXjfxrMsxqMHxoW/f1z6A/AzazPpiGTBS0kQASVtK2j+v2x5YKWlL4JTCNqvzupoVpNrA0HOC113A2cqX6pIO3PTw1zsp7/NQYFW+yp1PjlvS4aTyiI2Kp9R/nyGkal2Qbmv35kHgE5J2z8faMS9v5fe1AcQdtJk1LSLeJXWqF0laTHo2PSmv/h7wEDAXeLaw2bXAeTnxaQ/gp6RKTAtIz6C7M410u3hJTgSb1o9f5Y18/CuBM/KyqcB4SUuAC4HTu9n2XmC/WpIYcDHwT5IeIN3y71FE/C/wVeCGfA6vy6ta+X1tAHE1KzPbLEmaR6oB/minYzFrxFfQZmZmFeQraDMzswryFbSZmVkFuYM2MzOrIHfQZmZmFeQO2szMrILcQZuZmVXQ/wMXSKKNtXnhUwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "def plot_feature_importances_cancer(model):\n",
    "    n_features = cancer.data.shape[1]\n",
    "    plt.barh(range(n_features),model.feature_importances_,align='center')\n",
    "    plt.yticks(np.arange(n_features),cancer.feature_names)\n",
    "    plt.xlabel(\"Feature importance\")\n",
    "    plt.ylabel(\"Feature\")\n",
    "    \n",
    "plot_feature_importances_cancer(tree)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 可看出,worst radius是最好的划分属性"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 决策树总结:\n",
    "<p>控制决策树模型复杂度的参数是预剪枝参数，他在树完全展开之前停止对树的构造。通常来说,选择一种预剪枝策略(max_depth,meax_lef_nodes,min_samples_leafs)\n",
    "足以防止过拟合\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<strong>决策树优点:</strong>①得到的模型容易可视化②算法完全不受数据缩放的影响。由于每个特征被单独处理，而且数据的划分也不依赖于缩放\n",
    "<strong>决策树缺点:</strong>容易出现过拟合"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当特征的尺度完全不一样或者二元特征和连续特征同时存在时，决策树的效果很好"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 练一练.尝试使用决策树判断鸢尾花"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "鸢尾花前五条数据:\n",
      "[[5.1 3.5 1.4 0.2]\n",
      " [4.9 3.  1.4 0.2]\n",
      " [4.7 3.2 1.3 0.2]\n",
      " [4.6 3.1 1.5 0.2]\n",
      " [5.  3.6 1.4 0.2]]\n",
      "鸢尾花分类:\n",
      "[0 0 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.model_selection import train_test_split\n",
    "iris = load_iris()\n",
    "print(f'鸢尾花前五条数据:\\n{iris.data[:5]}')\n",
    "print(f\"鸢尾花分类:\\n{iris.target[:5]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集精度:1.0\n",
      "测试集精度:0.9736842105263158\n",
      "预测:[0]\n",
      "预测:['setosa']\n"
     ]
    }
   ],
   "source": [
    "X_train,X_test,y_train,y_test = train_test_split(iris.data,iris.target,random_state=0,stratify=iris.target)\n",
    "tree = DecisionTreeClassifier(random_state=0)\n",
    "tree.fit(X_train,y_train)\n",
    "print(f\"训练集精度:{tree.score(X_train,y_train)}\")\n",
    "print(f\"测试集精度:{tree.score(X_test,y_test)}\")\n",
    "import numpy as np\n",
    "X_new = np.array([[5,2.9,1,0.2]])\n",
    "prediction = tree.predict(X_new)\n",
    "print(f\"预测:{prediction}\")\n",
    "print(f\"预测:{iris['target_names'][prediction]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 练一练:根据已知数据判断红酒分类"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "红酒数据:[[1.423e+01 1.710e+00 2.430e+00 1.560e+01 1.270e+02 2.800e+00 3.060e+00\n",
      "  2.800e-01 2.290e+00 5.640e+00 1.040e+00 3.920e+00 1.065e+03]\n",
      " [1.320e+01 1.780e+00 2.140e+00 1.120e+01 1.000e+02 2.650e+00 2.760e+00\n",
      "  2.600e-01 1.280e+00 4.380e+00 1.050e+00 3.400e+00 1.050e+03]\n",
      " [1.316e+01 2.360e+00 2.670e+00 1.860e+01 1.010e+02 2.800e+00 3.240e+00\n",
      "  3.000e-01 2.810e+00 5.680e+00 1.030e+00 3.170e+00 1.185e+03]\n",
      " [1.437e+01 1.950e+00 2.500e+00 1.680e+01 1.130e+02 3.850e+00 3.490e+00\n",
      "  2.400e-01 2.180e+00 7.800e+00 8.600e-01 3.450e+00 1.480e+03]\n",
      " [1.324e+01 2.590e+00 2.870e+00 2.100e+01 1.180e+02 2.800e+00 2.690e+00\n",
      "  3.900e-01 1.820e+00 4.320e+00 1.040e+00 2.930e+00 7.350e+02]]\n",
      "红酒特征分类:['alcohol', 'malic_acid', 'ash', 'alcalinity_of_ash', 'magnesium', 'total_phenols', 'flavanoids', 'nonflavanoid_phenols', 'proanthocyanins', 'color_intensity', 'hue', 'od280/od315_of_diluted_wines', 'proline']\n",
      "红酒特征分类:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0\n",
      " 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1\n",
      " 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\n",
      " 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2]\n",
      "红酒标签:\n",
      "['class_0' 'class_1' 'class_2']\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_wine\n",
    "wine = load_wine()\n",
    "print(f\"红酒数据:{wine.data[:5]}\")\n",
    "print(f\"红酒特征分类:{wine.feature_names}\")\n",
    "print(f\"红酒特征分类:{wine.target}\")\n",
    "print(f\"红酒标签:\\n{wine.target_names}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集得分:1.0\n",
      "测试集得分:0.956\n"
     ]
    }
   ],
   "source": [
    "# 划分训练集和测试集\n",
    "X_train,X_test,y_train,y_test = train_test_split(wine.data,wine.target,stratify=wine.target,random_state=42)\n",
    "# 构造决策树\n",
    "wine_tree = DecisionTreeClassifier(random_state=0) # 防止平局的现象\n",
    "wine_tree.fit(X_train,y_train)\n",
    "print(f\"训练集得分:{round(wine_tree.score(X_train,y_train),3)}\")\n",
    "print(f\"测试集得分:{round(wine_tree.score(X_test,y_test),3)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "训练集得分:1.0\n",
      "测试集得分:0.956\n"
     ]
    }
   ],
   "source": [
    "# 设置最大深度为5\n",
    "wine_tree = DecisionTreeClassifier(random_state=0,max_depth=5) # 防止平局的现象\n",
    "wine_tree.fit(X_train,y_train)\n",
    "print(f\"训练集得分:{round(wine_tree.score(X_train,y_train),3)}\")\n",
    "print(f\"测试集得分:{round(wine_tree.score(X_test,y_test),3)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 可视化决策树\n",
    "from sklearn.tree import export_graphviz # 可视化决策树的方法\n",
    "export_graphviz(wine_tree,out_file='wine_tree.dot',class_names=['class_0','class_1','class_2'],feature_names=wine.feature_names,impurity=False,filled=True)\n",
    "import graphviz\n",
    "# 之后在目录命令行运行dot -Tpdf tree.dot -o tree.pdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
